{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "using VMLS\n",
    "using LinearAlgebra"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chapter 16\n",
    "# Constrained least squares\n",
    "### 16.1 Constrained least squares problem\n",
    "In the examples in this section, we use the `cls_solve` function, given later, to find the constrained least squares solution. \n",
    "\n",
    "**Piecewise polynomial.** We fit a function $f̂ : R → R$ to some given data, where $f̂(x) = p(x)$ for $x ≤ a$ and $f̂(x) = q(x)$ for $x > a$, subject to $p(a) = q(a)$ and $p′(a) = q′(a)$, i.e., the two polynomials have matching value and slope at the knot point $a$. We have data points $x1, . . . , x_M ≤ a$ and $x_{M+1}, . . . , x_N > a$ and corresponding values $y_1, . . . , y_N $. In the example we take $a = 0$, polynomials $p$ and $q$ of degree $3$, and $N = 2M = 140$. The code creates a figure similar to Figure [16.1](https://web.stanford.edu/~boyd/vmls/vmls.pdf#figure.16.1) of VMLS (Figure [16.1](#fig1)). We use the vandermonde function from page 58."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"600\" height=\"400\" viewBox=\"0 0 2400 1600\">\n",
       "<defs>\n",
       "  <clipPath id=\"clip7300\">\n",
       "    <rect x=\"0\" y=\"0\" width=\"2400\" height=\"1600\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<path clip-path=\"url(#clip7300)\" d=\"\n",
       "M0 1600 L2400 1600 L2400 0 L0 0  Z\n",
       "  \" fill=\"#ffffff\" fill-rule=\"evenodd\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip7301\">\n",
       "    <rect x=\"480\" y=\"0\" width=\"1681\" height=\"1600\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<path clip-path=\"url(#clip7300)\" d=\"\n",
       "M182.008 1487.47 L2352.76 1487.47 L2352.76 47.2441 L182.008 47.2441  Z\n",
       "  \" fill=\"#ffffff\" fill-rule=\"evenodd\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip7302\">\n",
       "    <rect x=\"182\" y=\"47\" width=\"2172\" height=\"1441\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polyline clip-path=\"url(#clip7302)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  243.444,1487.47 243.444,47.2441 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip7302)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  755.413,1487.47 755.413,47.2441 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip7302)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  1267.38,1487.47 1267.38,47.2441 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip7302)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  1779.35,1487.47 1779.35,47.2441 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip7302)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  2291.32,1487.47 2291.32,47.2441 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip7302)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  182.008,1359.35 2352.76,1359.35 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip7302)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  182.008,1091.15 2352.76,1091.15 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip7302)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  182.008,822.962 2352.76,822.962 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip7302)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  182.008,554.769 2352.76,554.769 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip7302)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  182.008,286.576 2352.76,286.576 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip7300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  182.008,1487.47 2352.76,1487.47 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip7300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  182.008,1487.47 182.008,47.2441 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip7300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  243.444,1487.47 243.444,1465.87 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip7300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  755.413,1487.47 755.413,1465.87 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip7300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1267.38,1487.47 1267.38,1465.87 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip7300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1779.35,1487.47 1779.35,1465.87 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip7300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  2291.32,1487.47 2291.32,1465.87 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip7300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  182.008,1359.35 214.569,1359.35 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip7300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  182.008,1091.15 214.569,1091.15 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip7300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  182.008,822.962 214.569,822.962 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip7300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  182.008,554.769 214.569,554.769 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip7300)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  182.008,286.576 214.569,286.576 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip7300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 243.444, 1541.47)\" x=\"243.444\" y=\"1541.47\">-1.0</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip7300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 755.413, 1541.47)\" x=\"755.413\" y=\"1541.47\">-0.5</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip7300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 1267.38, 1541.47)\" x=\"1267.38\" y=\"1541.47\">0.0</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip7300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 1779.35, 1541.47)\" x=\"1779.35\" y=\"1541.47\">0.5</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip7300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 2291.32, 1541.47)\" x=\"2291.32\" y=\"1541.47\">1.0</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip7300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 158.008, 1376.85)\" x=\"158.008\" y=\"1376.85\">-0.4</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip7300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 158.008, 1108.65)\" x=\"158.008\" y=\"1108.65\">-0.2</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip7300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 158.008, 840.462)\" x=\"158.008\" y=\"840.462\">0.0</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip7300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 158.008, 572.269)\" x=\"158.008\" y=\"572.269\">0.2</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip7300)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 158.008, 304.076)\" x=\"158.008\" y=\"304.076\">0.4</text>\n",
       "</g>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"491.492\" cy=\"418.473\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"491.492\" cy=\"418.473\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"702.858\" cy=\"367.972\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"702.858\" cy=\"367.972\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"384.914\" cy=\"492.752\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"384.914\" cy=\"492.752\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"448.54\" cy=\"455.598\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"448.54\" cy=\"455.598\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"974.665\" cy=\"310.076\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"974.665\" cy=\"310.076\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1246.28\" cy=\"223.889\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1246.28\" cy=\"223.889\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"420.283\" cy=\"459.285\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"420.283\" cy=\"459.285\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"983.411\" cy=\"218.075\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"983.411\" cy=\"218.075\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"724.371\" cy=\"177.387\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"724.371\" cy=\"177.387\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"951.341\" cy=\"336.092\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"951.341\" cy=\"336.092\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"458.652\" cy=\"270.496\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"458.652\" cy=\"270.496\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"629.873\" cy=\"308.397\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"629.873\" cy=\"308.397\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"429.623\" cy=\"503.57\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"429.623\" cy=\"503.57\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"736.611\" cy=\"173.362\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"736.611\" cy=\"173.362\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"387.564\" cy=\"494.987\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"387.564\" cy=\"494.987\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"732.856\" cy=\"417.067\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"732.856\" cy=\"417.067\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"371.849\" cy=\"463.326\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"371.849\" cy=\"463.326\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1055.97\" cy=\"377.475\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1055.97\" cy=\"377.475\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1121.28\" cy=\"196.413\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1121.28\" cy=\"196.413\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1188.41\" cy=\"219.467\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1188.41\" cy=\"219.467\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"778.874\" cy=\"170.073\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"778.874\" cy=\"170.073\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"660.625\" cy=\"223.753\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"660.625\" cy=\"223.753\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"901.087\" cy=\"278.556\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"901.087\" cy=\"278.556\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"550.3\" cy=\"297.538\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"550.3\" cy=\"297.538\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"926.142\" cy=\"312.327\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"926.142\" cy=\"312.327\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"446.171\" cy=\"429.409\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"446.171\" cy=\"429.409\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1073.81\" cy=\"243.706\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1073.81\" cy=\"243.706\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1205.59\" cy=\"261.666\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1205.59\" cy=\"261.666\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"812.029\" cy=\"88.0053\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"812.029\" cy=\"88.0053\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1059.5\" cy=\"247.501\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1059.5\" cy=\"247.501\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"674.792\" cy=\"278.815\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"674.792\" cy=\"278.815\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"714.34\" cy=\"240.696\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"714.34\" cy=\"240.696\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"572.794\" cy=\"218.106\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"572.794\" cy=\"218.106\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"396.227\" cy=\"456.917\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"396.227\" cy=\"456.917\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"856.836\" cy=\"335.927\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"856.836\" cy=\"335.927\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"600.523\" cy=\"197.902\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"600.523\" cy=\"197.902\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"876.514\" cy=\"279.416\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"876.514\" cy=\"279.416\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"400.614\" cy=\"503.201\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"400.614\" cy=\"503.201\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"968.499\" cy=\"189.884\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"968.499\" cy=\"189.884\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"510.662\" cy=\"406.89\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"510.662\" cy=\"406.89\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1153.04\" cy=\"375.78\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1153.04\" cy=\"375.78\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"685.188\" cy=\"210.625\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"685.188\" cy=\"210.625\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"521.586\" cy=\"424.914\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"521.586\" cy=\"424.914\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"371.828\" cy=\"511.205\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"371.828\" cy=\"511.205\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"487.839\" cy=\"290.731\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"487.839\" cy=\"290.731\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"343.306\" cy=\"689.407\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"343.306\" cy=\"689.407\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1065.28\" cy=\"270.055\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1065.28\" cy=\"270.055\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"308.301\" cy=\"665.596\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"308.301\" cy=\"665.596\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"297.773\" cy=\"606.494\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"297.773\" cy=\"606.494\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"332.513\" cy=\"709.984\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"332.513\" cy=\"709.984\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1231.9\" cy=\"191.604\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1231.9\" cy=\"191.604\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"293.974\" cy=\"670.42\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"293.974\" cy=\"670.42\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1207.06\" cy=\"328.13\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1207.06\" cy=\"328.13\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"433.383\" cy=\"463.379\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"433.383\" cy=\"463.379\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1247.7\" cy=\"213.97\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1247.7\" cy=\"213.97\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"817.171\" cy=\"298.911\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"817.171\" cy=\"298.911\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"304.197\" cy=\"598.8\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"304.197\" cy=\"598.8\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"783.624\" cy=\"278.344\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"783.624\" cy=\"278.344\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"424.234\" cy=\"352.535\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"424.234\" cy=\"352.535\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"851.179\" cy=\"312.614\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"851.179\" cy=\"312.614\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"525.892\" cy=\"428.617\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"525.892\" cy=\"428.617\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"620.301\" cy=\"376.794\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"620.301\" cy=\"376.794\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"260.725\" cy=\"792.499\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"260.725\" cy=\"792.499\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"777.408\" cy=\"251.962\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"777.408\" cy=\"251.962\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"976.336\" cy=\"332.754\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"976.336\" cy=\"332.754\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1170.97\" cy=\"380.73\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1170.97\" cy=\"380.73\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"495.4\" cy=\"321.29\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"495.4\" cy=\"321.29\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"915.121\" cy=\"239.364\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"915.121\" cy=\"239.364\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"699.62\" cy=\"269.733\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"699.62\" cy=\"269.733\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1009.07\" cy=\"271.084\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1009.07\" cy=\"271.084\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"2203.71\" cy=\"899.436\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"2203.71\" cy=\"899.436\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"2156.71\" cy=\"1012.92\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"2156.71\" cy=\"1012.92\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1588.98\" cy=\"1108.68\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1588.98\" cy=\"1108.68\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1657.06\" cy=\"1328.76\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1657.06\" cy=\"1328.76\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1414.5\" cy=\"616.031\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1414.5\" cy=\"616.031\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1809.08\" cy=\"1257.13\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1809.08\" cy=\"1257.13\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1694.42\" cy=\"1148.68\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1694.42\" cy=\"1148.68\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1303.94\" cy=\"238.759\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1303.94\" cy=\"238.759\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1313.84\" cy=\"241.634\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1313.84\" cy=\"241.634\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1826.78\" cy=\"1260.65\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1826.78\" cy=\"1260.65\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"2142.11\" cy=\"1028.69\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"2142.11\" cy=\"1028.69\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1482.89\" cy=\"758.216\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1482.89\" cy=\"758.216\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1633.69\" cy=\"1120.83\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1633.69\" cy=\"1120.83\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1287.01\" cy=\"381.911\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1287.01\" cy=\"381.911\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1309.14\" cy=\"344.851\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1309.14\" cy=\"344.851\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1717.31\" cy=\"1215.59\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1717.31\" cy=\"1215.59\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1597.2\" cy=\"1138.05\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1597.2\" cy=\"1138.05\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1635.19\" cy=\"1093.65\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1635.19\" cy=\"1093.65\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"2116.9\" cy=\"1107.53\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"2116.9\" cy=\"1107.53\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1763.59\" cy=\"1264.57\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1763.59\" cy=\"1264.57\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"2067.25\" cy=\"1195.18\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"2067.25\" cy=\"1195.18\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1952.88\" cy=\"1282.26\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1952.88\" cy=\"1282.26\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1490.93\" cy=\"917.054\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1490.93\" cy=\"917.054\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1334.82\" cy=\"292.032\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1334.82\" cy=\"292.032\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1612.54\" cy=\"1089.03\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1612.54\" cy=\"1089.03\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1824.54\" cy=\"1285.48\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1824.54\" cy=\"1285.48\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1856.4\" cy=\"1256.45\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1856.4\" cy=\"1256.45\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1874.8\" cy=\"1311.85\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1874.8\" cy=\"1311.85\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"2242.85\" cy=\"941.87\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"2242.85\" cy=\"941.87\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1530.22\" cy=\"815.821\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1530.22\" cy=\"815.821\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1968.28\" cy=\"1332.89\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1968.28\" cy=\"1332.89\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1331.43\" cy=\"234.957\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1331.43\" cy=\"234.957\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"2056.73\" cy=\"1181.31\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"2056.73\" cy=\"1181.31\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1800.69\" cy=\"1309.51\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1800.69\" cy=\"1309.51\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1715.66\" cy=\"1134.84\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1715.66\" cy=\"1134.84\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1309.2\" cy=\"296.683\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1309.2\" cy=\"296.683\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1809.69\" cy=\"1369.52\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1809.69\" cy=\"1369.52\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1791.21\" cy=\"1170.88\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1791.21\" cy=\"1170.88\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1956.45\" cy=\"1269.14\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1956.45\" cy=\"1269.14\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1343.76\" cy=\"481.595\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1343.76\" cy=\"481.595\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1726.83\" cy=\"1228.11\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1726.83\" cy=\"1228.11\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1596.81\" cy=\"1022.41\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1596.81\" cy=\"1022.41\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1976.15\" cy=\"1247.13\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1976.15\" cy=\"1247.13\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1880.55\" cy=\"1355.58\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1880.55\" cy=\"1355.58\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"2257.55\" cy=\"819.794\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"2257.55\" cy=\"819.794\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1427.85\" cy=\"713.744\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1427.85\" cy=\"713.744\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1814.73\" cy=\"1205.52\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1814.73\" cy=\"1205.52\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1306.12\" cy=\"247.617\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1306.12\" cy=\"247.617\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"2187.61\" cy=\"1082.81\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"2187.61\" cy=\"1082.81\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1497.6\" cy=\"932.216\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1497.6\" cy=\"932.216\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1507.14\" cy=\"888.979\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1507.14\" cy=\"888.979\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1409.11\" cy=\"629.088\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1409.11\" cy=\"629.088\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1847.83\" cy=\"1415.66\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1847.83\" cy=\"1415.66\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1880.08\" cy=\"1291.75\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1880.08\" cy=\"1291.75\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1309.77\" cy=\"331.314\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1309.77\" cy=\"331.314\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1562.05\" cy=\"1024.74\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1562.05\" cy=\"1024.74\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1735.45\" cy=\"1243.67\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1735.45\" cy=\"1243.67\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1821.9\" cy=\"1266.49\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1821.9\" cy=\"1266.49\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1503.09\" cy=\"877.877\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1503.09\" cy=\"877.877\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1526.35\" cy=\"940.431\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1526.35\" cy=\"940.431\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"2140.19\" cy=\"1199.79\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"2140.19\" cy=\"1199.79\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1356.56\" cy=\"492.154\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1356.56\" cy=\"492.154\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1820.43\" cy=\"1446.71\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1820.43\" cy=\"1446.71\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1700.03\" cy=\"1144.84\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1700.03\" cy=\"1144.84\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1377.88\" cy=\"555.883\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1377.88\" cy=\"555.883\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"2194.92\" cy=\"895.969\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"2194.92\" cy=\"895.969\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1719.78\" cy=\"1125.13\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1719.78\" cy=\"1125.13\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1475.44\" cy=\"765.317\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1475.44\" cy=\"765.317\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1684.59\" cy=\"1272.47\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1684.59\" cy=\"1272.47\" r=\"14\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#000000; stroke:none; fill-opacity:1\" cx=\"1431.75\" cy=\"681.251\" r=\"18\"/>\n",
       "<circle clip-path=\"url(#clip7302)\" style=\"fill:#009af9; stroke:none; fill-opacity:1\" cx=\"1431.75\" cy=\"681.251\" r=\"14\"/>\n",
       "<polyline clip-path=\"url(#clip7302)\" style=\"stroke:#e26f46; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  243.444,685.774 248.59,679.375 253.735,673.001 258.881,666.653 264.026,660.331 269.171,654.035 274.317,647.766 279.462,641.524 284.608,635.309 289.753,629.122 \n",
       "  294.898,622.963 300.044,616.833 305.189,610.732 310.335,604.66 315.48,598.618 320.626,592.606 325.771,586.624 330.916,580.673 336.062,574.753 341.207,568.865 \n",
       "  346.353,563.009 351.498,557.185 356.643,551.394 361.789,545.636 366.934,539.911 372.08,534.22 377.225,528.563 382.371,522.941 387.516,517.354 392.661,511.802 \n",
       "  397.807,506.286 402.952,500.806 408.098,495.362 413.243,489.955 418.388,484.585 423.534,479.253 428.679,473.959 433.825,468.703 438.97,463.486 444.116,458.308 \n",
       "  449.261,453.17 454.406,448.071 459.552,443.012 464.697,437.994 469.843,433.017 474.988,428.082 480.133,423.188 485.279,418.336 490.424,413.526 495.57,408.76 \n",
       "  500.715,404.036 505.861,399.356 511.006,394.72 516.151,390.129 521.297,385.582 526.442,381.08 531.588,376.624 536.733,372.213 541.878,367.849 547.024,363.531 \n",
       "  552.169,359.261 557.315,355.037 562.46,350.862 567.605,346.734 572.751,342.655 577.896,338.625 583.042,334.644 588.187,330.712 593.333,326.831 598.478,323 \n",
       "  603.623,319.219 608.769,315.49 613.914,311.812 619.06,308.186 624.205,304.612 629.35,301.091 634.496,297.623 639.641,294.208 644.787,290.847 649.932,287.539 \n",
       "  655.078,284.287 660.223,281.089 665.368,277.947 670.514,274.86 675.659,271.829 680.805,268.854 685.95,265.936 691.095,263.075 696.241,260.272 701.386,257.527 \n",
       "  706.532,254.839 711.677,252.211 716.823,249.641 721.968,247.131 727.113,244.68 732.259,242.29 737.404,239.96 742.55,237.691 747.695,235.483 752.84,233.337 \n",
       "  757.986,231.252 763.131,229.23 768.277,227.271 773.422,225.375 778.568,223.543 783.713,221.774 788.858,220.069 794.004,218.43 799.149,216.855 804.295,215.345 \n",
       "  809.44,213.901 814.585,212.524 819.731,211.213 824.876,209.968 830.022,208.791 835.167,207.682 840.313,206.64 845.458,205.667 850.603,204.763 855.749,203.928 \n",
       "  860.894,203.162 866.04,202.466 871.185,201.84 876.33,201.286 881.476,200.802 886.621,200.389 891.767,200.048 896.912,199.779 902.057,199.583 907.203,199.46 \n",
       "  912.348,199.41 917.494,199.433 922.639,199.531 927.785,199.703 932.93,199.949 938.075,200.271 943.221,200.668 948.366,201.142 953.512,201.691 958.657,202.317 \n",
       "  963.802,203.02 968.948,203.801 974.093,204.659 979.239,205.595 984.384,206.61 989.53,207.704 994.675,208.877 999.82,210.129 1004.97,211.462 1010.11,212.875 \n",
       "  1015.26,214.369 1020.4,215.944 1025.55,217.6 1030.69,219.339 1035.84,221.159 1040.98,223.063 1046.13,225.049 1051.27,227.119 1056.42,229.272 1061.57,231.51 \n",
       "  1066.71,233.832 1071.86,236.239 1077,238.731 1082.15,241.309 1087.29,243.973 1092.44,246.723 1097.58,249.56 1102.73,252.484 1107.87,255.496 1113.02,258.596 \n",
       "  1118.16,261.784 1123.31,265.06 1128.46,268.426 1133.6,271.881 1138.75,275.426 1143.89,279.061 1149.04,282.786 1154.18,286.603 1159.33,290.51 1164.47,294.51 \n",
       "  1169.62,298.601 1174.76,302.785 1179.91,307.061 1185.06,311.431 1190.2,315.894 1195.35,320.451 1200.49,325.103 1205.64,329.849 1210.78,334.69 1215.93,339.626 \n",
       "  1221.07,344.659 1226.22,349.787 1231.36,355.012 1236.51,360.334 1241.65,365.753 1246.8,371.27 1251.95,376.885 1257.09,382.598 1262.24,388.411 1267.38,394.322 \n",
       "  \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip7302)\" style=\"stroke:#3da44d; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1267.38,394.322 1272.53,400.365 1277.67,406.57 1282.82,412.935 1287.96,419.454 1293.11,426.127 1298.25,432.948 1303.4,439.915 1308.55,447.025 1313.69,454.273 \n",
       "  1318.84,461.658 1323.98,469.176 1329.13,476.822 1334.27,484.595 1339.42,492.491 1344.56,500.506 1349.71,508.638 1354.85,516.883 1360,525.237 1365.14,533.698 \n",
       "  1370.29,542.261 1375.44,550.925 1380.58,559.686 1385.73,568.539 1390.87,577.483 1396.02,586.514 1401.16,595.628 1406.31,604.822 1411.45,614.093 1416.6,623.438 \n",
       "  1421.74,632.853 1426.89,642.335 1432.04,651.881 1437.18,661.488 1442.33,671.152 1447.47,680.869 1452.62,690.638 1457.76,700.454 1462.91,710.314 1468.05,720.215 \n",
       "  1473.2,730.154 1478.34,740.126 1483.49,750.13 1488.63,760.162 1493.78,770.218 1498.93,780.295 1504.07,790.39 1509.22,800.5 1514.36,810.621 1519.51,820.751 \n",
       "  1524.65,830.885 1529.8,841.02 1534.94,851.154 1540.09,861.283 1545.23,871.403 1550.38,881.512 1555.53,891.606 1560.67,901.682 1565.82,911.736 1570.96,921.766 \n",
       "  1576.11,931.768 1581.25,941.738 1586.4,951.674 1591.54,961.572 1596.69,971.429 1601.83,981.241 1606.98,991.006 1612.12,1000.72 1617.27,1010.38 1622.42,1019.98 \n",
       "  1627.56,1029.52 1632.71,1039 1637.85,1048.41 1643,1057.75 1648.14,1067.02 1653.29,1076.2 1658.43,1085.31 1663.58,1094.34 1668.72,1103.27 1673.87,1112.12 \n",
       "  1679.02,1120.87 1684.16,1129.53 1689.31,1138.09 1694.45,1146.54 1699.6,1154.89 1704.74,1163.12 1709.89,1171.24 1715.03,1179.25 1720.18,1187.14 1725.32,1194.9 \n",
       "  1730.47,1202.54 1735.61,1210.05 1740.76,1217.42 1745.91,1224.66 1751.05,1231.76 1756.2,1238.71 1761.34,1245.53 1766.49,1252.19 1771.63,1258.69 1776.78,1265.05 \n",
       "  1781.92,1271.24 1787.07,1277.27 1792.21,1283.14 1797.36,1288.83 1802.51,1294.36 1807.65,1299.71 1812.8,1304.87 1817.94,1309.86 1823.09,1314.66 1828.23,1319.28 \n",
       "  1833.38,1323.7 1838.52,1327.93 1843.67,1331.95 1848.81,1335.78 1853.96,1339.4 1859.1,1342.81 1864.25,1346.02 1869.4,1349 1874.54,1351.77 1879.69,1354.32 \n",
       "  1884.83,1356.64 1889.98,1358.73 1895.12,1360.59 1900.27,1362.22 1905.41,1363.61 1910.56,1364.76 1915.7,1365.67 1920.85,1366.32 1926,1366.73 1931.14,1366.88 \n",
       "  1936.29,1366.78 1941.43,1366.41 1946.58,1365.78 1951.72,1364.88 1956.87,1363.71 1962.01,1362.27 1967.16,1360.55 1972.3,1358.55 1977.45,1356.27 1982.59,1353.7 \n",
       "  1987.74,1350.84 1992.89,1347.68 1998.03,1344.23 2003.18,1340.48 2008.32,1336.42 2013.47,1332.06 2018.61,1327.39 2023.76,1322.4 2028.9,1317.1 2034.05,1311.48 \n",
       "  2039.19,1305.54 2044.34,1299.27 2049.49,1292.67 2054.63,1285.73 2059.78,1278.46 2064.92,1270.85 2070.07,1262.9 2075.21,1254.61 2080.36,1245.96 2085.5,1236.96 \n",
       "  2090.65,1227.6 2095.79,1217.89 2100.94,1207.81 2106.08,1197.37 2111.23,1186.56 2116.38,1175.37 2121.52,1163.81 2126.67,1151.87 2131.81,1139.55 2136.96,1126.85 \n",
       "  2142.1,1113.75 2147.25,1100.27 2152.39,1086.39 2157.54,1072.1 2162.68,1057.42 2167.83,1042.34 2172.98,1026.84 2178.12,1010.93 2183.27,994.611 2188.41,977.871 \n",
       "  2193.56,960.71 2198.7,943.123 2203.85,925.109 2208.99,906.664 2214.14,887.783 2219.28,868.466 2224.43,848.706 2229.57,828.503 2234.72,807.852 2239.87,786.75 \n",
       "  2245.01,765.193 2250.16,743.179 2255.3,720.705 2260.45,697.766 2265.59,674.36 2270.74,650.483 2275.88,626.132 2281.03,601.305 2286.17,575.996 2291.32,550.204 \n",
       "  \n",
       "  \"/>\n",
       "</svg>\n"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "M = 70; N = 2*M;\n",
    "xleft = rand(M) .- 1; xright = rand(M);\n",
    "x = [xleft; xright]\n",
    "y = x.^3 - x + 0.4 ./ (1 .+ 25*x.^2) + 0.05*randn(N);\n",
    "n = 4;\n",
    "A = [ vandermonde(xleft,n) zeros(M,n); \n",
    "    zeros(M,n) vandermonde(xright,n)]\n",
    "b = y;\n",
    "C = [1 zeros(1,n-1) -1 zeros(1,n-1); \n",
    "    0 1 zeros(1,n-2) 0 -1 zeros(1,n-2)];\n",
    "d = zeros(2);\n",
    "theta = cls_solve(A, b, C, d);\n",
    "using Plots\n",
    "# Evaluate and plot for 200 equidistant points on each side.\n",
    "Npl = 200;\n",
    "xpl_left = linspace(-1, 0, Npl);\n",
    "ypl_left = vandermonde(xpl_left, 4)*theta[1:n];\n",
    "xpl_right = linspace(0, 1, Npl);\n",
    "ypl_right = vandermonde(xpl_right, 4)*theta[n+1:end];\n",
    "scatter(x,y, legend=false)\n",
    "plot!(xpl_left, ypl_left)\n",
    "plot!(xpl_right, ypl_right)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Figure 16.1** Least squares fit of two cubic polynomials to $140$ points, with\n",
    "continuity constraints $p(0) = q(0)$ and $p′(0) = q′(0)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Advertising budget.** We continue the advertising example of page 96 and add a\n",
    "total budget constraint $1^T s = 1284$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Recall from chapter 12:\n",
    "R = [0.97 1.86 0.41;\n",
    "    1.23 2.18 0.53;\n",
    "    0.80 1.24 0.62;\n",
    "    1.29 0.98 0.51;\n",
    "    1.10 1.23 0.69;\n",
    "    0.67 0.34 0.54;\n",
    "    0.87 0.26 0.62;\n",
    "    1.10 0.16 0.48;\n",
    "    1.92 0.22 0.71;\n",
    "    1.29 0.12 0.62];\n",
    "m, n = size(R);\n",
    "vdes = 1e3 * ones(m);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Array{Float64,1}:\n",
       " 315.16818459234986\n",
       " 109.86643348012271\n",
       " 858.9653819275275 "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cls_solve(R, vdes, ones(1,n), [1284])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Minimum norm force sequence.** We compute the smallest sequence of ten forces, each applied for one second to a unit frictionless mass originally at rest, that moves the mass position one with zero velocity (VMLS page [343](https://web.stanford.edu/~boyd/vmls/vmls.pdf#section*.373)). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2×10 Array{Float64,2}:\n",
       " 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0\n",
       " 9.5  8.5  7.5  6.5  5.5  4.5  3.5  2.5  1.5  0.5"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = eye(10); b = zeros(10);\n",
    "C = [ones(1,10); (9.5:-1:0.5)']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10-element Array{Float64,1}:\n",
       "  0.05454545454545475  \n",
       "  0.04242424242424237  \n",
       "  0.030303030303030096 \n",
       "  0.0181818181818182   \n",
       "  0.0060606060606059305\n",
       " -0.006060606060606132 \n",
       " -0.01818181818181816  \n",
       " -0.030303030303030318 \n",
       " -0.04242424242424239  \n",
       " -0.05454545454545448  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d = [0,1];\n",
    "fln = cls_solve(A,b,C,d)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 16.2 Solution\n",
    "Let’s implement the function `cls_solve_kkt`, which finds the constrained least squares solution by forming the $KKT$ system and solving it. We allow the `b` and `d` to be matrices, so one function call can solve mutiple problems with the same $A$ and $C$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2×1 Array{Float64,2}:\n",
       " -4.440892098500626e-16\n",
       " -3.885780586188048e-16"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function cls_solve_kkt(A,b,C,d)\n",
    "    m, n = size(A)\n",
    "    p, n = size(C)\n",
    "    G = A'*A # Gram matrix\n",
    "    KKT = [2*G C'; C zeros(p,p)] # KKT matrix; Karush–Kuhn–Tucker\n",
    "    xzhat = KKT \\ [2*A'*b; d]\n",
    "    return xzhat[1:n,:]\n",
    "end;\n",
    "A = randn(10,5); b = randn(10);\n",
    "C = randn(2,5); d = randn(2);\n",
    "x = cls_solve_kkt(A,b,C,d);\n",
    "C*x - d # Check residual small"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 16.3 Solving constrained least squares problems\n",
    "**Solving constrained least squares via QR.** Let’s implement VMLS algorithm [16.1](https://web.stanford.edu/~boyd/vmls/vmls.pdf#algorithmctr.16.1) and then check it against our method above, which forms and solves the $KKT$ system."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```julia\n",
    "function cls_solve(A,b,C,d)\n",
    "    m, n = size(A)\n",
    "    p, n = size(C)\n",
    "    Q, R = qr([A; C])\n",
    "    Q = Matrix(Q)\n",
    "    Q1 = Q[1:m,:]\n",
    "    Q2 = Q[m+1:m+p,:]\n",
    "    Qtil, Rtil = qr(Q2')\n",
    "    Qtil = Matrix(Qtil)\n",
    "    w = Rtil \\ (2*Qtil'*Q1'*b - 2*(Rtil'\\d))\n",
    "    return xhat = R \\ (Q1'*b - Q2'*w/2)\n",
    "end\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5.855045090355223e-16"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check with KKT method\n",
    "m = 10; n = 5; p = 2;\n",
    "A = randn(m,n); b = randn(m); C = randn(p,n); d = randn(p);\n",
    "xKKT = cls_solve_kkt(A,b,C,d);\n",
    "xQR = cls_solve(A,b,C,d);\n",
    "norm(xKKT-xQR)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function `cls_solve` is included in the `VMLS` package.\n",
    "\n",
    "**Sparse constrained least squares.** Let’s form and solve the system of linear equations in VMSL ([16.11](https://web.stanford.edu/~boyd/vmls/vmls.pdf#equation.16.3.11)), and compare it to our basic method for constrained least squares. This formulation will result in a sparse set of equations to solve if $A$ and $C$ are sparse. (The code below just checks that the two methods agree; it does not use sparsity. Unlike the earlier `cls_solve`, it assumes `b` and `d` are vectors.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6.670680158494424e-15"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function cls_solve_sparse(A,b,C,d)\n",
    "    m, n = size(A)\n",
    "    p, n = size(C)\n",
    "    bigA = [ zeros(n,n) A' C';\n",
    "        A -I/2 zeros(m,p) ;\n",
    "        C zeros(p,m) zeros(p,p) ]\n",
    "    xyzhat = bigA \\ [zeros(n) ; b ; d]\n",
    "    return xhat = xyzhat[1:n]\n",
    "end\n",
    "m = 100; n = 50; p = 10;\n",
    "A = randn(m,n); b = randn(m); C = randn(p,n); d = randn(p);\n",
    "x1 = cls_solve(A,b,C,d);\n",
    "x2 = cls_solve_sparse(A,b,C,d);\n",
    "norm(x1-x2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Solving least norm problem.** In Julia, the backslash operator is used to find the\n",
    "least norm solution $x̂$ of an under-determined set of equations $Cx = d$. Thus the\n",
    "backslash operator is overloaded to solve linear equations with a square coefficient\n",
    "matrix, find a least squares approximate solution when the coefficient matrix is\n",
    "tall, and find the least norm solution when the coefficient matrix is wide.\n",
    "\n",
    "Let’s solve a least norm problem using several methods, to check that they\n",
    "agree."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.274547555887833e-15"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p = 50; n = 500;\n",
    "C = randn(p,n); d = randn(p);\n",
    "x1 = C\\d; # Solve using backslash\n",
    "# Solve using cls_solve, which uses KKT system\n",
    "x2 = cls_solve(eye(n), zeros(n), C, d);\n",
    "x3 = pinv(C)*d; # Using pseudo-inverse\n",
    "norm(x1-x2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.3143371247983843e-15"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "norm(x2-x3)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Julia 1.1.1",
   "language": "julia",
   "name": "julia-1.1"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.1.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
